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DETAILED ACTION 

1 . The Examiner notes tliat claims 6, 7, 10, and 22 liave been cancelled without 
prejudice and claims 23-26 have been added via the amendment filed 2/22/2008. 
Claims 1-5, 8-9, 11-21, and 23-26 are pending. 

Response to Arguments 

2. Applicant's arguments filed 2/22/2008 have been fully considered but they are 
not persuasive. 

3. Re claims 1,16, and 21 , the Applicant contends that the prior art cited fails to 
teach or suggest the claimed feature in which the system transforms wide-angle data 
into both a panoramic view and a virtual view. However, the Examiner respectfully 
disagrees. Koyanagi discloses generating a panoramic view and the user utilizing the 
generated panoramic view to control the pan tilter camera, thus controlling the operation 
area/virtual view (Koyanagi: paragraph [0043]). Therefore, the data received from the 
camera portion is used in the generation of both of the panoramic view and the 
operation area/virtual view. 

4. Re claim 14, the Applicant contends that the prior art cited fails to teach or 
suggest that the reference window on the panoramic view, which indicates the outline of 
the virtual view, has a size and position determined according to user command data. 
However, the Examiner respectfully disagrees. Koyanagi discloses generating a 
panoramic view and the user utilizing the generated panoramic view to control the pan 
tilter camera, thus controlling the operation area/virtual view (Koyanagi: paragraph 
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[0043]). Furthermore, Koyanagi discloses that the zoom operation changes the size of 
the reference window within the panoramic view (Koyanagi: Fig. 1 6B; Figs. 1 0A and 
10B; paragraph [0094]). 

5. Applicant's arguments with respect to claims 9 and 19 have been considered but 
are moot in view of the new ground(s) of rejection. 

6. Re claim 1 1 , the Applicant contends that the prior art cited does not teach or 
suggest that the system transforms wide-angle image data received by the buffer into 
virtual view data corresponding to at least one virtual view and into panoramic view data 
corresponding to at least one panoramic view in real time (emphasis added by 
Applicant). However, the Examiner respectfully disagrees. Koyanagi discloses the flow 
of the control algorithm advances to relevant steps in accordance with a user input, 
indicating real-time processing (Koyanagi: Fig. 15; paragraph [0131]). Real time 
processing conventionally refers to executing a processing step as quickly as possible 
upon the triggering of an event (i.e., user input). Therefore, quickly generating a 
panoramic image upon the user selecting the panoramic generation button qualifies as 
real time processing. 

7. Re claim 15, the Applicant contends that the prior art cited fails to teach or 
suggest that the remaining portion of the monitored area is displayed in the second 
panoramic view, thus displaying the full 360° viewing area. However, the Examiner 
respectfully disagrees. Poelstra discloses registering of complete surroundings 
in a single image with the help of a fish eye optic (Poelstra: column 1 , lines SC- 
SI), wherein successive radial panoramic images are generated from the original 
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fish eye image (Poelstra: column 1, line 65, through column 2, line 7). By 
disclosing that the fish eye image registers the complete surroundings, Poelstra 
indicates that the fish eye image encompasses 360°, and, thus, one of ordinary 
skill in the art at the time of the invention would have found it obvious that the 
radial images generated from the original fish eye image would also encompass 
the complete surroundings. Although Poelstra does not explicitly state that the 
original fish eye image is divided into two radial panoramic images, one of 
ordinary skill in the art at the time of the invention would have found it obvious 
that it would be possible to divide the fish eye image into two successive 
panoramic images encompassing the complete surroundings in accordance with 
the resolution requirements of a display monitor for displaying very wide 
panoramic images, so as to meet the system requirements of the system 
designer. 

8. Re claim 15, the Applicant further contends that the prior art cited falls to teach or 
suggest an angle of reference for the viewing area. However, the Examiner respectfully 
disagrees. Poelstra discloses that the fish eye image is converted to panoramic images 
with reference to the center of the fish eye image (Poelstra: column 2, lines 1-2), 
wherein the center of the fish eye image corresponds to a reference axis, and an angle 
gamma indicates the maximum viewing angle with respect to an axis that is 
perpendicular to the axis corresponding to the center of the fish eye image (Poelstra: 
Fig. 7; column 3, lines 56-61). 
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9. A modified copy of the previous rejection, reflecting the changes made to the 
claims via the amendment filed 2/22/2008, is included below. 



Claim Rejections - 35 USC § 102 

1 0. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

11. Claims 1^, 8, 11-14, 16-18, and 20, 21, 23, and 26 are rejected under 35 
U.S.C. 102(e) as being anticipated by Koyanagi et al. (US 20040257436 A1). 

Re claim 1, Koyanagi discloses a system for creating signals indicative of a 
graphical user interface from wide-angle image data corresponding to a monitored area, 
said system comprising: a buffer configured to receive wide-angle image data 
corresponding to the monitored area (Koyanagi: Fig. 3, storing portion 30); and a 
processor (Koyanagi: Fig. 3, computer 1 and controlling portion 31) operably coupled to 
said buffer and configured to: transform wide angle image data received by the buffer 
into: panoramic view data corresponding to at least one panoramic view of the 
monitored area; and virtual view data corresponding to at least one virtual view of a 
portion of the at least one panoramic view (Koyanagi: paragraph [0012]); and encode at 
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least one panoramic image and at least one virtual view from the wide angle image data 
for display (Koyanagi: paragraph [0043], captured images are displayed; paragraph 
[0051], pictures may be captured in a particular format). 

Re claim 2, Koyanagi discloses a user input module configured to provide user 
command data to said processor (Koyanagi: Fig. 3, pointing device 14; paragraph 

[0045]); and said processor being further configured to determine the virtual view data 
based on the user command data (Koyanagi: paragraph [0044]). 

Re claim 3, Koyanagi discloses that the processor is further configured to 
determine reference data corresponding to an area in the panoramic view represented 
by the virtual view (Koyanagi: Fig. 1 ; paragraph [0043], "a frame 6C that represents the 
current position and the angle of view of the pan filter and a pan filter limiter 6D are 
superimposed to the panorama picture"). 

Re claim 4, Koyanagi discloses a first video camera system operably coupled to 
said buffer and said processor (Koyanagi: Fig. 3, lens block 15 and CCD 19 generate 
image data), said first video camera system operable to generate wide-angle image 
data (Koyanagi: Figs. 4A-4F and paragraphs [0021] and [0053]-[0054], image data is 
compiled into a panoramic view). 

Re claim 8, Koyanagi discloses that said processor performs operations on the 
wide-angle image data to correct distortion in the wide-angle image data, such that said 
panoramic view and said virtual view are corrected images (Koyanagi: paragraph 
[0054]). 



Application/Control Number: 10/802,451 Page 7 

Art Unit: 2621 

Re claim 11, Koyanagi discloses that the system transforms wide-angle image 
data received by the buffer into virtual view data corresponding to at least one virtual 
view and into panoramic view data corresponding to at least one panoramic view in real 
time (Koyanagi: Fig. 15; paragraph [0131], the flow of the control algorithm advances to 
relevant steps in accordance with a user input, indicating real-time processing). 

Re claim 12, Koyanagi discloses a display device operably coupled to said 
processor to display the at least one panoramic view and the at least one virtual view 
(Koyanagi: Fig. 3, monitor 2). 

Re claim 13, Koyanagi discloses that the at least one panoramic view 
corresponds to a substantially undistorted view of the monitored area (Koyanagi: 
paragraphs [0053]-[0054], generating undistorted panoramic view), and the at least one 
virtual view corresponds to a portion of the at least one panoramic view (Koyanagi: 
paragraph [0012]). 

Re claim 14, Koyanagi discloses at least one reference window overlaid on at 
least one portion of the at least one panoramic view, each overlaid portion defining the 
portion of the at least one panoramic view to which the at least one virtual view 
corresponds (Koyanagi: Figs. 1 and 2), and the at least one reference window having a 
size and a position determined according to the user command data (Koyanagi: 
paragraph [0043], user controls position of operation area/virtual view; Fig. 16B; Figs. 
10A and 10B; paragraph [0094], zoom operation changes the size of the reference 
window within the panoramic view). 
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Claim 16 recites tlie corresponding method for implementation within the system 
of claim 1 , and, therefore, has been analyzed and rejected with respect to claim 1 
above. 

Re claim 17, Koyanagi discloses determining pan, tilt and zoom values 
(Koyanagi: Fig. 16A, step SI 3); and determining the portions of the buffered data to 
transform into virtual view data for at least one virtual view based on the pan, tilt and 
zoom values (Koyanagi: paragraphs [0050]-[0051]). 

Re claim 18, Koyanagi discloses determining reference data based on the pan, 
tilt and zoom values (Koyanagi: paragraph [0043]). 

Re claim 20, Koyanagi discloses encoding reference data, virtual view data and 
panoramic view data for output (Koyanagi: paragraph [0051]). 

Re claim 21, Koyanagi discloses a system for creating signals indicative of a 
graphical user interface from wide-angle image data corresponding to a monitored area, 
said system comprising: means for storing wide-angle image data corresponding to a 
monitored area (Koyanagi: Fig. 3, storing portion 30); and means for processing and 
transforming said wide-angle image data received from said storing means into 
panoramic view data corresponding to at least one panoramic view of the monitored 
area and into virtual view data corresponding to at least one virtual view of a portion of 
the at least one panoramic view (Koyanagi: paragraph [0012]; paragraphs [0053]- 
[0054], images merged to form panoramic pictures). 
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Re claim 23, Koyanagi discloses that the size and the position of each reference 
window determines pan, tilt and zoom values for the corresponding virtual view 
(Koyanagi: paragraph [0043], user controls position of operation area/virtual view; Fig. 
16B; Figs. 10A and 10B; paragraph [0094], zoom operation changes the size of the 
reference window within the panoramic view). 

Re claim 26, Koyanagi discloses determining a position and a size of at least 
one reference window positioned over the portion of at least one of the at least one 
panoramic view corresponding to the virtual view, the position and size defined 
according to user command data (Koyanagi: paragraph [0043], user controls position of 
operation area/virtual view; Fig. 16B; Figs. 10A and 108; paragraph [0094], zoom 
operation changes the size of the reference window within the panoramic view); and 
wherein the pan, tilt and zoom values are based upon the position and the size of the at 
least one reference window (Koyanagi: paragraph [0044], user input controls pan filter). 

Claim Rejections - 35 USC § 103 

1 2. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

13. Claims 9 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Koyanagi et al. (US 20040257436 A1) in view of Monroe (US 20070182819 
A1). 
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Re claim 9, Koyanagi discloses a majority of the features of claim 9, as 
discussed above in claim 4, and additionally a camera system operably coupled to the 
processor (Koyanagi: Fig. 3, camera portion 11), said camera system having a camera 
and being configured to aim the camera at a portion of the monitored area according to 
pan, tilt and zoom command data (Koyanagi: paragraphs [0048]-[0049]), and configured 
to capture video image data (Koyanagi: Fig. 3, camera portion 1 1 outputs a video 
signal); and wherein the processor is further configured to communicate pan, tilt and 
zoom command data to cause the camera system to aim the camera at a portion of the 
monitored area (Koyanagi: paragraphs [0048]-[0049]), but Koyanagi does not 
specifically disclose that the camera system includes a second video camera; and 
wherein captured video image data from the second video camera is included in the at 
least one virtual view. However, Monroe discloses a digital security multimedia sensor 
system, wherein panoramic views of monitored areas are created by merging images 
captured by a multitude of cameras (Monroe: Fig. 2, elements 10a-10h) and cameras 
within the array may be selectively scrutinized in order to track objects (Monroe: Fig. 16; 
paragraph [0106]). Since both Koyanagi and Monroe relate to panoramic image 
processing, one of ordinary skill in the art at the time of the invention would have found 
it obvious to combine the multiple cameras of Monroe with the photographing system of 
Koyanagi in order to permit transmission of the least amount of data to accomplish the 
desired image transmission (Monroe: paragraph [0024]). The combined system of 
Koyanagi and Monroe has all of the features of claim 9. 
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Re claim 19, the combined system of Koyanagi and Poelstra discloses a majority 
of the features of claim 19, as discussed in claims 16-18 above. Koyanagi additionally 
discloses that the buffered wide-angle data is received from a first video camera system 
communicating pan, tilt and zoom commands to a camera system (Koyanagi: paragraph 
[0049]); and receiving virtual view data for the at least one virtual view (Koyanagi: 
paragraph [0012]) from the camera system (Koyanagi: paragraph [0012]), but Koyanagi 
does not specifically disclose a second virtual view and receiving virtual view data for 
the second virtual view, wherein captured video image data from the second video 
camera is included in the at least one virtual view. However, Monroe discloses a digital 
security multimedia sensor system, wherein panoramic views of monitored areas are 
created by merging images captured by a multitude of cameras (Monroe: Fig. 2, 
elements lOa-IOh) and cameras within the array may be selectively scrutinized in order 
to track objects (Monroe: Fig. 16; paragraph [0106]). Since both Koyanagi and Monroe 
relate to panoramic image processing, one of ordinary skill in the art at the time of the 
invention would have found it obvious to combine the multiple cameras of Monroe with 
the photographing system of Koyanagi in order to permit transmission of the least 
amount of data to accomplish the desired image transmission (Monroe: paragraph 
[0024]). The combined system of Koyanagi and Monroe has all of the features of claim 
19. 
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14. Claims 5, 15, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Koyanagi et al. (US 20040257436 A1) in view of Poelstra (US 
5563650 A). 

Re claim 5, Koyanagi discloses a majority of the features of claim 5, as 
discussed above in claim 4, but Koyanagi does not specifically disclose that the video 
camera system includes a fisheye lens. However, Poelstra discloses a device for 
producing and consulting panoramic images, wherein images are produced using a fish 
eye lens and the images are transformed into panoramic images (Poelstra: column 1 , 
lines 42-48). Since both Koyanagi and Poelstra relate to producing panoramic images 
for review by a user, one of ordinary skill in the art at the time of the invention would 
have found it obvious to combine the fish eye lens of Poelstra with the photographing 
system of Koyanagi in order to produce a system capable of creating multiple 
panoramic views very quickly for consultation by the user (Poelstra: column 1, lines 38- 
41 ). The combined system of Koyanagi and Poelstra has all of the features of claim 5. 

Re claim 15, Koyanagi disclose a majority of the features of claim 15, as 
discussed in claim 14 above. Additionally, Koyanagi discloses that the at least one 
panoramic view includes a first panoramic view, the first panoramic view corresponding 
to a first portion of the monitored area (Koyanagi: Figs. 4A-4E, a panoramic image 
representing a portion of the monitoring area is extracted); the at least one virtual view 
includes a first virtual view, the first virtual view corresponding to a first portion of the 
first panoramic view (Koyanagi: Figs. 1 and 2, the operation area represents a selected 
portion of the panorama operation area), but Koyanagi does not specifically disclose 
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that the at least one panoramic view includes a second panoramic view, the second 
panoramic view corresponding to the remaining portion of the monitored area, and a 
second virtual view, the second virtual view corresponding to a second portion of the 
second panoramic view; and a combination of the first panoramic view and the second 
panoramic view provide a 360° view of the monitored area relative to a vertical 
axis. However, Poelstra discloses a device for producing and consulting panoramic 
images, wherein images are produced using a fish eye lens and the images are 
transformed into panoramic images (Poelstra: column 1, lines 42-48), and more than 
one panoramic image may be extracted from the initial fish eye image (Poelstra: Figs. 5 
and 6; column 3, lines 42-46). Poelstra further discloses registering of complete 
surroundings in a single image with the help of a fish eye optic (Poelstra: column 
1, lines 50-51), wherein successive radial panoramic images are generated from 
the original fish eye image (Poelstra: column 1 , line 65, through column 2, line 7). 
By disclosing that the fish eye image registers the complete surroundings, 
Poelstra indicates that the fish eye image encompasses 360°, and, thus, one of 
ordinary skill in the art at the time of the invention would have found it obvious 
that the radial images generated from the original fish eye image would also 
encompass the complete surroundings. Poelstra also discloses that the fish eye 
image is converted to panoramic images with reference to the center of the fish eye 
image (Poelstra: column 2, lines 1-2), wherein the center of the fish eye image 
corresponds to a reference axis, and an angle gamma indicates the maximum viewing 
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angle with respect to an axis tliat is perpendicular to the axis corresponding to the 
center of the fish eye image (Poelstra: Fig. 7; column 3, lines 56-61 ). Since both 
Koyanagi and Poelstra relate to producing images for consultation by a user, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the multiple panoramic image display of Poelstra with the panoramic/virtual 
view display of Koyanagi in order to produce a system capable of producing and 
consulting a large number of panoramic views quickly (Poelstra: column 1, lines 38-41). 
The combined system of Koyanagi and Poelstra has all of the features of claim 15. 

Re claim 24, the combined system of Koyanagi and Poelstra discloses a majority 
of the features of claim 24, as discussed above in claim 15. Additionally, Koyanagi 
discloses that the at least one reference window is user-selectable for controlling the 
size and the position of the reference window to determine pan, tilt and zoom values for 
the corresponding virtual view (Koyanagi: paragraph [0043], user controls position of 
operation area/virtual view; Fig. 16B; Figs. 10A and 10B; paragraph [0094], zoom 
operation changes the size of the reference window within the panoramic view). 

Re claim 25, Koyanagi discloses a majority of the features of claim 25, as 
discussed above in claim 16. Koyanagi does not specifically disclose two panoramic 
views. However, Poelstra discloses a device for producing and consulting panoramic 
images, wherein the at least one panoramic view includes: a first panoramic view 
corresponding to a first portion of the monitored area (Poelstra: Fig. 6, transformed 
image 22; column 3, lines 3-4 and 42-46); a second panoramic view corresponding to a 
remaining portion of the monitored area (Poelstra: Fig. 6, transformed image 23; column 
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3, lines 3-4 and 42-46); and the at least two panoramic views combine to provide a 360° 
view of the monitored area relative to a vertical axis (Poelstra: column 1 , lines 50-51 , 
registering of complete surroundings in a single image with the help of a fish eye 
optic; column 1 , line 65, through column 2, line 7, successive radial panoramic 
images are generated from the original fish eye image); and wherein said method 
further comprises encoding the at least two panoramic views and at least one virtual 
view of a portion of at least one of the at least two panoramic views for simultaneous 
display (Poelstra: Fig. 6; column 3, lines 3-4, two panoramic images are displayed 
simultaneously). Since both Koyanagi and Poelstra relate to producing images for 
consultation by a user, one of ordinary skill in the art at the time of the invention would 
have found it obvious to combine the multiple panoramic image display of Poelstra with 
the panoramic/virtual view display of Koyanagi in order to produce a system capable of 
producing and consulting a large number of panoramic views quickly (Poelstra: column 
1 , lines 38-41 ). The combined system of Koyanagi and Poelstra has all of the features 
of claim 25. 

Conclusion 

15. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

a. Image reproduction apparatus with panoramic mode based on aspect ratio 

Suzuki (US 720601 7 B1) 
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b. Multi-Sensor Panoramic Network Camera 
Kaplinsl<y (US 20050141607 A1) 

c. Visual user interface for use in controlling the interaction of a device with a 
spatial region 

Lasslter(US 6624846 B1) 

d. System for a plurality of video cameras disposed on a common network 
Metzger et al. (US 20060136972 A1) 

e. Surround surveillance system for mobile body, and mobile body, car, and 
train using the same 

Kumata et al. (US 20020005896 Al) 

f. Surround surveillance apparatus for mobile body 
Kumata et al. (US 2002008001 7 Al ) 

16. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, liowever, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 



Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
(571 )270-1 1 99. The examiner can normally be reached on Monday-Friday 8:30am- 
5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha D. Banks-Harold can be reached on (571 ) 272-7905. The fax 
phone number for the organization where this application or proceeding is assigned is 
571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Marsha D. Banks-Harold/ 

Supervisory Patent Examiner, Art Unit 2621 

/Christopher Findley/ 
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